Papers with human reading
Towards Multi-Modal Text-Image Retrieval to improve Human Reading (2021.naacl-srw)
Copied to clipboard
| Challenge: | In primary school, children's books, as well as in modern language learning apps, multi-modal learning strategies like illustrations of terms and phrases are used to support reading comprehension. |
| Approach: | They propose to use multi-modal transformers to train multi-dimensional models on text-image retrieval to support a user's reading comprehension of arbitrary text. |
| Outcome: | The proposed model performs poorly because of the short and relatively simple textual data that the current models are trained with. |
Joint Multi-Label Attention Networks for Social Text Annotation (N19-1)
Copied to clipboard
| Challenge: | Present research shows that title metadata could affect social annotation. |
| Approach: | They propose a title-guided attention network for document annotation with user-generated tags that separates the title from the content of a document and applies a semantic-based loss regulariser over each sentence in the content. |
| Outcome: | The proposed approach outperforms the Bi-GRU and Hierarchical Attention Network (HAN) on two open datasets with 10%-30% reduction in training time. |
A Neural Model of Adaptation in Reading (D18-1)
Copied to clipboard
| Challenge: | Several studies suggest that readers do adapt their lexical and syntactic predictions to the current context. |
| Approach: | They propose to add a simple adaptation mechanism to a neural language model to improve predictions of reading times. |
| Outcome: | The proposed model improves predictions of human reading times compared to a non-adaptive model. |
Probing for Reading Times (2026.acl-long)
Copied to clipboard
Eleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re, Tianyang Xu, Mario Giulianelli, Karolina Stanczak, Ryan Cotterell
| Challenge: | a large body of work on probing has demonstrated that language model representations encode a wealth of linguistic information, but it remains unclear whether they also capture cognitive signals about human processing. |
| Approach: | They use regularized linear regression to compare language model representations against scalar predictors. |
| Outcome: | The representations from early layers outperform surprisal in predicting early-pass measures such as first fixation and gaze duration. |
How to Engage your Readers? Generating Guiding Questions to Promote Active Reading (2024.acl-long)
Copied to clipboard
| Challenge: | Using questions in written text is an effective strategy to enhance readability, but what makes an active reading question good, what the linguistic role of these questions is, and what is their impact on human reading remains understudied. |
| Approach: | They present a dataset of 10K in-text questions from textbooks and scientific articles and explore various approaches to generate such questions using language models. |
| Outcome: | The generated questions are of high quality and are almost as effective as human-written questions in terms of improving readers’ memorization and comprehension. |
An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal (2026.acl-long)
Copied to clipboard
| Challenge: | Surprisal theory claims that difficulty of sentences increases linearly with surprise . a neural LM that can explain garden-path effects cannot be built, says a new study . |
| Approach: | They propose to fine-tune neural LMs to better align surprisal-based reading-time estimates with actual reading times. |
| Outcome: | a new study shows that fine-tuned neural LMs do not overfit on held-out items . the results show that they improve predictive power for human reading times . |
Towards A Scanpath-Conditioned Surprisal Theory: Modeling Reader Information States (2026.acl-long)
Copied to clipboard
| Challenge: | Standard surprisal is computed from the linear text prefix, but human reading is non-linear and memory constrained. |
| Approach: | They propose a formulation of surprisal conditioned on a reader-specific accessible information state given by the scanpath history and memory dynamics rather than by the written prefix alone. |
| Outcome: | The proposed approach improves on eye-tracking measures on the written prefix and on eye movement data on human reading. |